Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[nexus][saga] Fix saga node pagination, use Paginator API #5953

Merged
merged 5 commits into from
Jun 28, 2024

Conversation

smklein
Copy link
Collaborator

@smklein smklein commented Jun 25, 2024

  • Uses the Paginator API for listing sagas and saga node events
  • For saga node events, paginates by (node_id, event_type) rather than by saga_id, which was previously incorrect
  • Adds regression tests that cross the BATCH_SIZE boundary and check pagination order for both queries

Fixes #5948

@smklein smklein marked this pull request as ready for review June 26, 2024 18:21
@smklein smklein requested a review from davepacheco June 26, 2024 18:21
@@ -103,54 +99,4 @@ impl DataStore {
)),
}
}

pub async fn saga_list_unfinished_by_id(
Copy link
Collaborator Author

@smklein smklein Jun 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that both these functions are only called once, and are already part of the db-queries crate, I condensed them into saga_recovery.rs.

This made it marginally easier to specify arguments, and means we can re-use the same pool connection for each batch we access.

This was mostly for a minor convenience, but I can keep the old structure if you had plans to actually call these functions from elsewhere.

@smklein smklein changed the title [nexus][saga] Paginate saga nodes by node ID, not saga ID [nexus][saga] Fix saga node pagination, use Paginator API Jun 26, 2024
nexus/db-queries/src/db/saga_recovery.rs Outdated Show resolved Hide resolved
nexus/db-queries/src/db/saga_recovery.rs Outdated Show resolved Hide resolved
nexus/db-queries/src/db/saga_recovery.rs Outdated Show resolved Hide resolved
Comment on lines 322 to 325
ErrorHandler::NotFoundByLookup(
ResourceType::SagaDbg,
LookupType::ById(saga.id.0 .0),
),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is what the old code was doing but I don't think it's right? It may not matter. But if we get no rows here, that's not an error at all.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a test for this case, I think the "no node, no saga in db" case still returns a vec of length zero, so I'm pretty sure this is dead code. Updated to ErrorHandler::Server to make this slightly more apparent.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to be super sure, I did a quick check and it looks to me like the ErrorHandler::NotFound* variants only do anything special if the returned Diesel error variant is itself NotFound, which only happens if we use first() (first_async() for us) or get_result() (get_result_async() for us). We're using load_async(), and the NotFound docs explicitly say load() doesn't consider it an error to get 0 rows. So that explains why it didn't matter and yes I think the update you made makes sense. Thanks!

Copy link
Collaborator

@davepacheco davepacheco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@smklein smklein merged commit 886548b into main Jun 28, 2024
19 checks passed
@smklein smklein deleted the saga-pagination branch June 28, 2024 00:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

saga recovery failed due to incomplete log due to pagination bug
2 participants